<!DOCTYPE html>
<html xmlns:MadCap="http://www.madcapsoftware.com/Schemas/MadCap.xsd" class="_Skins_HTML5___Top_Navigation" lang="en-us" xml:lang="en-us" data-mc-search-type="Stem" data-mc-help-system-file-name="Default.xml" data-mc-path-to-help-system="../../" data-mc-has-content-body="True" data-mc-target-type="WebHelp2" data-mc-runtime-file-type="Topic;Default" data-mc-preload-images="false" data-mc-in-preview-mode="false" data-mc-toc-path="">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <meta charset="utf-8" />
        <meta http-equiv="X-UA-Compatible" content="IE=edge" />
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <link href="../../Skins/Default/Stylesheets/Slideshow.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/TextEffects.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Topic.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Styles.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Tablet.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Mobile.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/foundation.6.2.3.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Styles.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Tablet.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Mobile.css" rel="stylesheet" type="text/css" data-mc-generated="True" /><title>Working with Template Packs</title>
        <link href="../Resources/Stylesheets/Styles.css" rel="stylesheet" type="text/css" />
        <script src="../../Resources/Scripts/custom.modernizr.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/jquery.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/require.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/require.config.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/foundation.6.2.3_custom.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/plugins.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapGlobal.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapDom.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapUtilities.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapXhr.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapTextEffects.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapSlideshow.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapFeedback.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapDefault.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapHelpSystem.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.Breadcrumbs.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.MiniToc.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.SideMenu.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapIndex.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapGlossary.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapParser.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapSearch.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapTopic.js" type="text/javascript">
        </script>
    </head>
    <body>
        <div class="foundation-wrap off-canvas-wrapper">
            <div class="off-canvas-wrapper-inner" data-off-canvas-wrapper="">
                <aside class="off-canvas position-left" id="offCanvas" data-off-canvas="" data-position="left" data-mc-ignore="true">
                    <ul class="off-canvas-drilldown vertical menu off-canvas-list" data-drilldown="" data-mc-back-link="Back" data-mc-css-tree-node-expanded="is-drilldown-submenu-parent" data-mc-css-tree-node-collapsed="is-drilldown-submenu-parent" data-mc-css-sub-menu="vertical menu slide-in-left is-drilldown-submenu" data-mc-include-indicator="False" data-mc-include-icon="False" data-mc-include-parent-link="True" data-mc-include-back="True" data-mc-defer-expand-event="True" data-mc-expand-event="click.zf.drilldown" data-mc-toc="True">
                    </ul>
                </aside>
                <div class="off-canvas-content inner-wrap" data-off-canvas-content="">
                    <div data-sticky-container="" class="title-bar-container">
                        <nav class="title-bar tab-bar sticky" data-sticky="" data-options="marginTop:0" style="width:100%" data-sticky-on="only screen and (max-width: 1279px)" data-mc-ignore="true">
                            <div class="middle title-bar-section outer-row clearfix">
                                <div class="menu-icon-container relative clearfix">
                                    <button class="menu-icon" data-toggle="offCanvas"><span></span>
                                    </button>
                                </div>
                            </div>
                            <div class="title-bar-layout outer-row">
                                <div class="logo-wrapper"><a class="logo" href="index.htm" alt="Logo"></a>
                                </div>
                                <div class="navigation-wrapper nocontent">
                                    <ul class="navigation clearfix" data-mc-css-tree-node-has-children="has-children" data-mc-css-sub-menu="sub-menu" data-mc-expand-event="mouseenter" data-mc-top-nav-menu="True" data-mc-max-depth="3" data-mc-include-icon="False" data-mc-include-indicator="False" data-mc-include-children="True" data-mc-include-siblings="True" data-mc-include-parent="True" data-mc-toc="True">
                                        <li class="placeholder" style="visibility:hidden"><a>placeholder</a>
                                        </li>
                                    </ul>
                                </div>
                                <div class="nav-search-wrapper">
                                    <div class="nav-search row">
                                        <form class="search" action="#">
                                            <div class="search-bar search-bar-container needs-pie">
                                                <input class="search-field needs-pie" type="search" placeholder="Search" />
                                                <div class="search-filter-wrapper">
                                                    <div class="search-filter">
                                                        <div class="search-filter-content">
                                                            <ul>
                                                                <li>All Files</li>
                                                            </ul>
                                                        </div>
                                                    </div>
                                                </div>
                                                <div class="search-submit-wrapper" dir="ltr">
                                                    <div class="search-submit" title="Search">
                                                    </div>
                                                </div>
                                            </div>
                                        </form>
                                    </div>
                                </div>
                            </div>
                        </nav>
                    </div>
                    <section class="main-section">
                        <div class="row outer-row sidenav-layout">
                            <div class="sidenav-wrapper">
                                <div class="sidenav-container">
                                    <ul class="off-canvas-accordion vertical menu sidenav" data-accordion-menu="" data-mc-css-tree-node-expanded="is-accordion-submenu-parent" data-mc-css-tree-node-collapsed="is-accordion-submenu-parent" data-mc-css-sub-menu="vertical menu accordion-menu is-accordion-submenu nested" data-mc-include-indicator="False" data-mc-include-icon="False" data-mc-include-parent-link="False" data-mc-include-back="False" data-mc-defer-expand-event="True" data-mc-expand-event="click.zf.accordionMenu" data-mc-toc="True" data-mc-side-nav-menu="True">
                                    </ul>
                                </div>
                            </div>
                            <div class="body-container" data-mc-content-body="True">
                                <h1>Working with Template Packs</h1>
                                <ul>
                                    <li value="1"><a href="#What" class="MCXref xref">What is a Template Pack?</a>
                                    </li>
                                    <li value="2"><a href="#The" class="MCXref xref">The Pack Anatomy</a>
                                    </li>
                                    <li value="3"><a href="#Provided" class="MCXref xref">Provided Template Packs</a>
                                    </li>
                                    <li value="4"><a href="#Customiz" class="MCXref xref">Customizing the Provided Packs</a>
                                    </li>
                                    <li value="5"><a href="#Creating" class="MCXref xref">Creating a New Template Pack</a>
                                    </li>
                                    <li value="6"><a href="#Placehol" class="MCXref xref">Placeholders in values.yaml </a> </li>
                                </ul>
                                <h2><a name="What"></a>What is a Template Pack?</h2>
                                <p>Every experiment run on the Intel® Deep Learning Studio application utilizes a template pack. For each experiment, a template pack defines the experiment’s complete runtime environment and any supporting infrastructure required to run that experiment.</p>
                                <p>Each template pack includes a number of elements or templates that together define the Kubernetes* (K8s) application, which executes a user-provided experiment script based on specific supporting technology.</p>
                                <p>Each template pack includes templates that define a Dockerfile, the Kubernetes service definition, deployments, jobs, a configuration map and any other standard Kubernetes elements needed to create a runtime environment for an experiment instance.</p>
                                <p style="text-align: center;">
                                    <img src="../images/template_pack.png" style="border-left-style: solid;border-left-width: 2px;border-right-style: solid;border-right-width: 2px;border-top-style: solid;border-top-width: 2px;border-bottom-style: solid;border-bottom-width: 2px;" />
                                </p>
                                <p>Individual elements within a pack are referred to as templates because they contain a number of placeholders (some are required, while others are optional), that are substituted with appropriate values by Intel DL Studio software during experiment submission. These placeholders define items such as: experiment name, user namespace, the address of the local Intel DL Studio Docker registry, and other variables that may change between different experiment runs.</p>
                                <p>The core Kubernetes definitions within each pack are grouped into <a href="https://helm.sh/">Helm</a>* packages referred to as <i>Charts</i>. Helm is the de-facto standard for Kubernetes application packaging, and reusing this package format allows leveraging of the large resource of community-developed Helm charts when creating new Intel DL Studio template packs.</p>
                                <p><b>Note 1</b>: While Intel DL Studio is able to re-use Helm charts mostly verbatim, there are a number of required placeholders that need to be added to these charts for Intel DL Studio to track and manage the resulting experiments. See Creating new packs below for details.</p>
                                <p><b>Note 2</b>:  All supported Intel DL Studio template packs are distributed with the nctl package.</p>
                                <h2><a name="The"></a>The Pack Anatomy</h2>
                                <h3>Location</h3>
                                <p>When the <span style="font-family: 'Courier New';">nctl</span> package is installed on the client machine, the template packs that come with the official package are deposited in the folder:</p>
                                <p><span style="font-family: 'Courier New';">NCTL_HOME/config/packs</span>
                                </p>
                                <p>Each pack resides in a dedicated sub-folder, named after the pack.</p>
                                <h3>The Pack Folder Structure</h3>
                                <p><a name="packs"></a>The individual items that form a single pack are laid out in its folder as follows:</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">&lt;PACK_NAME&gt;/</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">Dockerfile</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">charts/</p>
                                <p style="font-family: 'Courier New';text-indent: 1.5in;">Chart.yaml</p>
                                <p style="font-family: 'Courier New';text-indent: 1.5in;">values.yaml</p>
                                <p style="font-family: 'Courier New';text-indent: 1.5in;">templates/</p>
                                <p>Where:</p>
                                <ul>
                                    <li value="1"><a name="Dockerfile"></a><span style="font-family: 'Courier New';">Dockerfile</span> is the Docker file that defines the Docker image which serves as the runtime for the experiment's script supplied by the user. Any dependencies needed to build the Docker image must be placed in this directory, next to the Dockerfile.</li>
                                    <li value="2"><a name="Charts"></a><span style="font-family: 'Courier New';">Charts</span> is a directory that hosts the Helm chart that specifies the definitions of all Kubernetes entities used to deploy and support the experiment's Docker image in the cluster.</li>
                                    <li value="3"><a name="Chart_yaml"></a><span style="font-family: 'Courier New';">Chart.yaml</span> provides the key metadata for the chart, such as name and version, and about the chart.</li>
                                    <li value="4"><a name="values_yaml"></a><span style="font-family: 'Courier New';">values.yaml</span> serves a key role as it provides definitions for various Helm template placeholders (see Helm's <i><a href="https://docs.helm.sh/chart_template_guide/">Chart Template Guide</a></i> for details) used throughout the chart (mostly in the individual Kubernetes definitions contained within the templates sub-folder). This file is also parsed and analyzed by <span style="font-family: 'Courier New';">nctl</span> to perform substitution on <a href="#Placehol" class="MCXref xref">Placeholders in values.yaml </a>.</li>
                                    <li value="5">
                                        <p><a name="templates"></a><span style="font-family: 'Courier New';">templates</span> folder groups all the YAML files that provide definitions for various Kubernetes (K8s) entities, which define the packs deployment and runtime environment. These definitions are referred to as templates as they may include Helm template placeholders substituted for actual values in the process of deploying the chart on the cluster.</p>
                                    </li>
                                </ul>
                                <h2><a name="Provided"></a>Provided Template Packs</h2>
                                <p>The following template packs are provided with Intel DL Studio.:</p>
                                <p>(<b>Note to author</b>:&#160;<i>The following list needs to be updated with Beta Release</i>.)</p>
                                <ul>
                                    <li value="1"><span style="font-family: 'Courier New';">multinode-tf-training-tfjob</span>:&#160;A TensorFlow multi-node training job based on TF-operator using Python 3.</li>
                                    <li value="2"><span style="font-family: 'Courier New';">multinode-tf-training-tfjob-py2</span>:&#160;A TensorFlow multi-node training job based on TF-operator using Python 2.</li>
                                    <li value="3"><span style="font-family: 'Courier New';">tf-training-tfjob</span>:&#160;A TensorFlow single-node training job based on TF-operator using Python 3.</li>
                                    <li value="4">t<span style="font-family: 'Courier New';">f-training-tfjob-py2</span>:&#160;A TensorFlow single-node training job based on TF-operator using Python 2.</li>
                                </ul>
                                <h2><a name="Customiz"></a>Customizing the Provided Packs</h2>
                                <p>Any customizations to template packs revolve mostly around the <span style="font-family: 'Courier New';">values.yaml</span> file included in the pack's underlying Helm chart. As mentioned in <a href="#The2" class="MCXref xref">Working with Template Packs</a>, this file provides key definitions that are referenced throughout the rest of the Helm chart, and therefore it plays a crucial role in the process of converting the chart's templates into actual Kubernetes definitions deployed on the cluster.</p>
                                <p>By convention, the definitions contained in the <span style="font-family: 'Courier New';">values.yaml</span> file typically reference parameters that are intended to be customized by end-users, so in most cases it is safe to manipulate those without corrupting the pack.</p>
                                <p><b>Note</b>: This is in contrast to parameters not intended for customization. In addition, these parameters typically live within the templates themselves.</p>
                                <h3>Altering Parameters Listed in values.yaml</h3>
                                <p>When altering parameters listed in the <span style="font-family: 'Courier New';">values.yaml</span> file, there are two approaches:</p>
                                <ol>
                                    <li value="1">Users may manually modify the pack's <span style="font-family: 'Courier New';">values.yaml</span> file using a text editor. Any modifications done using this approach will be permanent and apply to all subsequent experiments based on this pack.</li>
                                    <li value="2">Users may alter some of the parameters listed in <span style="font-family: 'Courier New';">values.yaml</span> file temporarily and <i>only</i> for a single experiment. To do so, the user may specify alternative values for any of the parameters listed in <span style="font-family: 'Courier New';">values.yaml</span> using the <span style="font-family: 'Courier New';">--pack_param</span> option when submitting an experiment (refer to <a href="view_cli_help.htm">CLI&#160;Commands</a> for more details).</li>
                                </ol>
                                <p>Advanced users who want full control over how their experiments are deployed and executed on the Kubernetes cluster may also directly modify the templates residing in the <span style="font-family: 'Courier New';">&lt;PACK_NAME&gt;/charts/templates/</span> folder. Doing this, however requires a good grasp of Kubernetes concepts, notation, and debugging techniques, and is therefore not recommended.</p>
                                <h2><a name="Creating"></a>Creating a New Template Pack</h2>
                                <h4>Prerequisites</h4>
                                <p>Creating a new pack, while not overly complex, requires some familiarity with the technologies that packs are built on. Therefore, it is recommended to have at least some working experience in the following areas do this:</p>
                                <ul>
                                    <li value="1">Creating/modifying Helm charts and specifically using the <a href="https://docs.helm.sh/chart_template_guide/">Helm templates</a>.</li>
                                    <li value="2">Defining and managing Kubernetes entities such as <span style="font-family: 'Courier New';">pods</span>, <span style="font-family: 'Courier New';">jobs</span>, <span style="font-family: 'Courier New';">deployments</span>, <span style="font-family: 'Courier New';">services</span>, etc.</li>
                                </ul>
                                <h4>Where to Start</h4>
                                <p>Creating new template packs for Intel DL Studio is greatly simplified by leveraging the relatively ubiquitous Helm chart format as the foundation.</p>
                                <p>Thus the starting point for a new template pack is typically an existing Helm chart that packages the technology of choice for execution on a K8s cluster. Consider creating a chart from scratch only if an existing chart <i>is not</i> available. The process of creating a new Helm chart from scratch is exhaustively described in Helm <a href="https://docs.helm.sh/chart_template_guide/">documentation</a>.</p>
                                <h3>A Template Pack in Five Simple Steps</h3>
                                <p>Once a working Helm chart is available, the process of adapting it for use as an Intel DL Studio template is as follows:</p>
                                <ol>
                                    <li value="1">Pick the pack's name. The name should be unique and not conflict with any other packs available in the local <a href="#packs">packs</a> folder. After naming the pack, create a corresponding directory in the packs folder and populate its <a href="#Charts">charts</a> subfolder with the contents of the chart.</li>
                                    <li value="2">Create a <a href="#Dockerfile">Dockerfile</a>. This Dockerfile will be used to build the image that will host the experiment's scripts. As such, it should include all libraries and other dependencies that experiments based on this pack will use at runtime.</li>
                                    <li value="3">Update <span style="font-family: 'Courier New';">values.yaml</span> (or create it if it does not exist). The following items that must be placed in the chart's <a href="#values_yaml">values.yaml</a> file in order to enable proper experiment tracking:</li>
                                </ol>
                                <ul>
                                    <li value="1">The <span style="font-family: 'Courier New';">podCount</span> element must be defined and initialized with the expected number of experiment pods that must enter the Running state in order for Intel DL Studio to consider the experiment as started.</li>
                                    <li value="2">If the experiment script to be used with the pack accepts any command-line arguments, then a <span style="font-family: 'Courier New';">commandlin</span>e parameter must be specified and assigned the value of <a href="#NAUTA.Co" class="MCXref xref">NAUTA.CommandLine</a> placeholder. This will allow the commandline parameters specified in the <span style="font-family: 'Courier New';">nctl experiment submit</span> command to be propagated to the relevant Helm chart elements (by referencing the 'commandline' parameter specified in <span style="font-family: 'Courier New';">values.yaml</span>)</li>
                                    <li value="3">An <span style="font-family: 'Courier New';">image</span> parameter must be specified and assigned the value of <a href="#NAUTA.Ex" class="MCXref xref">NAUTA.ExperimentImage</a>. The actual name of this parameter does not matter as long as it is properly referenced wherever a container image for the experiment is specified within the chart templates.</li>
                                </ul>
                                <ol start="4">&gt;
            <li value="4">Add tracking labels. The <span style="font-family: 'Courier New';">podCount</span> element specified above indicates how many pods to expect within a normally functioning experiment based on this pack. The way Intel DL Studio identifies the pods that belong to particular experiment is based on specific labels that need to be assigned to each pod that <i>should</i> be included in the <span style="font-family: 'Courier New';">podCount</span> number. The label in question is <span style="font-family: 'Courier New';">runName</span> and it needs to be assigned the value corresponding to the name of the current Helm release (by assigning the Helm <span style="font-family: 'Courier New';">{{ .Release.name }}</span> template placeholder).</li></ol>
                                <p><b>Note</b>: Not all pods within an experiment need to be accounted for in <span style="font-family: 'Courier New';">podCount</span> and assigned the aforementioned label. Intel DL Studio only needs to track the pods in which the runtime state is representative of  the overall experiment status. If, for instance, an experiment is composed of a "master" pod which in turn manages its fleet of worker pods, then its sufficient to set <span style="font-family: 'Courier New';">podCount</span> to 1 and only track the "master" as long as it's state (<i>Pending, Running, Failed,</i> etc) is representative for the entire group.</p>
                                <ol start="5">
                                    <li value="5">Update container image references. All container image definitions with the chart's templates that need to point to the image running the experiment script (as defined in the <span style="font-family: 'Courier New';">Dockerfile</span> in step #1) need to refer to the corresponding <span style="font-family: 'Courier New';">image</span> Helm template placeholder as previously defined in <span style="font-family: 'Courier New';">values.yaml</span> (step #3 above).</li>
                                </ol>
                                <h2><a name="Placehol"></a>Placeholders in values.yaml </h2>
                                <h4><a name="NAUTA.Co"></a>NAUTA.CommandLine</h4>
                                <p>The NAUTA.CommandLine placeholder, when placed within the <span style="font-family: 'Courier New';">values.yaml</span> file, will be substituted for the list of command line parameters specified when submitting an experiment via <span style="font-family: 'Courier New';">nctl experiment submit</span> command.</p>
                                <p>To pass this list as the command line into one of the containers defined in the pack's templates, it needs to be first assigned to a parameter within <span style="font-family: 'Courier New';">values.yaml</span> This parameter then needs to be referenced within the chart's templates just like any other Helm template parameter.</p>
                                <p>The following example snippet shows the placeholder being used to initialize a parameter named: commandline:</p>
                                <p style="font-family: 'Courier New';">commandline:</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">args:</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">{% for arg in NAUTA.CommandLine %}nt</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">- {{ arg }}</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">{% endfor %}</p>
                                <h4><a name="NAUTA.Ex"></a>NAUTA.ExperimentImage</h4>
                                <p>The NAUTA.ExperimentImage placeholder carries the full reference to the docker image resulting from building the Dockerfile specified within the pack.</p>
                                <p>During experiment submission the image will be built by Docker and deposited in the NAUTA Docker Registry under the locator represented by this placeholder.</p>
                                <p>Hence, the placeholder shall be used to initialize a template parameter within the <span style="font-family: 'Courier New';">values.yaml</span> file, that will later be referenced within the chart's templates to specify the experiment image.</p>
                                <p>Below is a sample definition of a parameter within <span style="font-family: 'Courier New';">values.yaml</span>, followed by a sample reference to the image in pod template.</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">&lt;values.yaml&gt;</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">image: {{ NAUTA.ExperimentImage }}</p>
                                <p style="font-family: 'Courier New'; text-indent: 0.5in;">&#160;</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">&lt;pod.yaml&gt;</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">containers:</p>
                                <p style="font-family: 'Courier New';text-indent: 0.5in;">- name: tensorflow</p>
                                <p style="font-family: 'Courier New';text-indent: 1in;">image: "{{ .Values.image }}"</p>
                            </div>
                        </div>
                    </section><a data-close="true"></a>
                </div>
            </div>
            <script>/* <![CDATA[ */$(document).foundation();/* ]]> */</script>
        </div>
    </body>
</html>